kraco.dev

Sign In
Analyzing complex video using Gemini 2.5 in The Bazaar

I've been playing a new video game called The Bazaar recently (would highly recommend if you enjoy rogue-likes and deck building), and occasionally I record screen captures or photos of board states that I find particularly interesting.

Out of curiosity, I fed one of my own gameplay clips into Gemini 2.5 and asked it to explain what was going on — including a detailed breakdown of every action trigger.

I'll attach the results, but suffice to say: I was impressed.

There are definitely hallucinations written as confident statements, which is unfortunate, but the sheer number of things it got right made me appreciate just how much progress AI video analysis has made in the past year.

Asynchronous PvP Done Right

One of the most fascinating aspects of The Bazaar is that it's not real-time PvP. When you challenge another player, the battle is simulated ahead of time based on both board states, and then replayed client-side.

This means:

It makes each match feel like programming a machine — you assemble your build, press play, and watch it unfold.

Modeling trigger resolution

Look, no sane person should ever model a reactive battle simulation using `useEffect` — and yet, here we are.

This is not "good" React. This is deeply cursed React. But if you're the type to see an asynchronous combat engine and think _"hmm, I wonder if I can do this with hooks" — welcome to the club.

Here's a very dumbed-down, very readable version of what a trigger-resolution engine might look like in a React app:

TYPESCRIPT
1import { useState, useEffect } from 'react';
2

3function useBattleSimulation(players) {
4  const [queue, setQueue] = useState([]);
5  const [log, setLog] = useState([]);
6

7  useEffect(() => {
8    if (queue.length === 0) return;
9

10    const timer = setTimeout(() => {
11      const [next, ...rest] = queue;
12      next?.execute();
13      setLog((prev) => [...prev, next]);
14      const newTriggers = getTriggeredActions(players);
15      setQueue([...rest, ...newTriggers]);
16    }, 100);
17

18    return () => clearTimeout(timer);
19  }, [queue, players]);
20

21  const startBattle = (initialActions) => setQueue(initialActions);
22

23  return { log, startBattle };
24}
25

26function getTriggeredActions(players) {
27  return players.flatMap(player =>
28    player.items.flatMap(item =>
29      item.triggers
30        .filter(trigger => trigger.condition())
31        .map(trigger => trigger.action())
32    )
33  );
34}

This isn't a physics engine. This is a turn-based Rube Goldberg machine where each action says "hello" to five other things, who all then scream back "me too!" and jump into the queue. It's dumb. It's delightful. It's The Bazaar.

My Build: Haste, Poison, and the Deadly Tortuga

In the match I analyzed, I built around the "Aquatic" and "Friend" tags. My hero was Vanessa, and the core idea was to create a self-reinforcing engine of:

Key items were:

I also had perks like:

Watching it all unfold felt like seeing a machine come alive.

At one point, Tortuga hit for over 3400 crit damage, and then did it again a few seconds later — before the opponent had even resolved all their own item activations.

What Gemini 2.5 got right (and wrong)

Gemini correctly traced many of the major interactions — calling out which effects triggered others, explaining how the skills amplified my build, and narrating the cascade of Haste buffs and Poison ticks.

But it also confidently hallucinated a few things. Here's a breakdown of where it drifted from reality, with links to specific lines in the transcript:

All that said, I’m still impressed. The errors were mostly based on missing visual context, and the overall logical structure — what triggered what — held together surprisingly well.

Conclusion

This kind of tooling could easily evolve into:

And beyond gaming, it reflects a big leap in how well AI can parse visual sequences governed by invisible state changes.

For now, I'm just thrilled that something I built — a synergistic, time-sensitive trigger machine — could be broken down and mostly understood by a model like Gemini.

If nothing else, it’s a fun reminder that the line between gameplay and programming has never been thinner.

Gemini Transcript

Initial State (0:00)

Player: Pirate Queen Hero (Level 17), 7070 Health. Items (Left to Right): Deadly Tortuga, Jellyfish, Heavy Pufferfish, Icy Jellyfish, Heavy Electric Eels. All items have aquatic and friend tags. Tortuga and Eels are weapons. Pufferfish is medium size; others are small/large.

Opponent: Pirate Captain Hero (Level 26), Health TBD. Items: Sword, Octopus, Binoculars, Spear, Shield.

Game Modifier (Revealed Later): Glass Cannon — Both players' weapons have double damage.

Player Perks (Revealed Later):

Key Item Synergies (Revealed Later):

Combat Begins (0:01–0:08)

0:01 Combat starts. Opponent health: 10005. Timer begins.

0:02

0:03

0:04

0:05

0:06

0:07

0:08 Opponent defeated.

Post-Combat (0:09–0:22)

0:09–0:10: Victory screen. Player health: 1196. 0:11–0:12: Glass Cannon revealed. 0:13–0:22: Player reviews perks and items — confirms full synergy stack.

Summary

The player won using a fast-paced, synergistic build focused on Aquatic and Friend tags, leveraging Haste and Charge mechanics:

This feedback loop caused rapid item cycling, constant status effects, and overwhelming weapon damage — securing victory before the opponent's own Glass Cannon boosts could take full effect.

Hire me!